Statistical Hypothesis Testing Based on Machine Learning: Large Deviations Analysis

We study the performance of Machine Learning (ML) classification techniques. Leveraging theory large deviations, we provide mathematical conditions for a ML classifier to exhibit error probabilities that vanish exponentially, say $\exp (-n\,I)$ , where notation="LaTeX">$n$ is number informative observations available testing (or another relevant parameter, such as size target in an image) and notation="LaTeX">$I$ rate. Such depend on Fenchel-Legendre transform cumulant-generating function Data-Driven Decision Function (D3F, i.e., what thresholded before final binary decision made) learned training phase. As such, D3F related rate given set. The exponential convergence can be verified tested numerically exploiting dataset or synthetic generated according underlying statistical model. Coherently with deviations theory, also establish normalized statistic Gaussian distribution. Furthermore, approximate probability curves notation="LaTeX">$\zeta _{n} \exp are provided, thanks refined asymptotic derivation, _{n}$ represents most representative sub-exponential terms probabilities. asymptotic, able compute accurate analytical approximation both regimes small values . Theoretical findings corroborated by extensive numerical simulations use real-world data, acquired X-band maritime radar system surveillance.